Fractional quantum Hall effect of topological surface states under a strong tilted 

magnetic field 
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The fractional quantum Hall effect (FQHE) of topological surface-state particles under a tilted 
strong magnetic field is theoretically studied by using the exact diagonalization method. The Hal- 
dane's pseudopotentials for the Coulomb interaction are analytically obtained. The results show 
that by increasing the in-plane component of the tilted magnetic field, the FQHE state at n=0 Lan- 
dau level (LL) becomes more stable, while the stabilities of n=±l LLs become weaker. Moreover, 
we find that the excitation gaps of the v = 1/3 FQHE states increase as the tilt angle is increased. 
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The fractional quantum Hall effect (FQHE) in con- 
ventional tow-dimensional electron gas (2DEG) has been 
studied intensively in the last 30 years PHD] because of 
its rich physical properties. Recently, the unconventional 
sequence of FQHE in graphene, where the valley isospin 
combined with the usual electron spin yields fourfold de- 
generate Landau levels (LLs), has been observed pj-ll-j. 
Differing from the case of the conventional 2DEG, due 
to the Dirac nature of the electrons and the additional 
symmetry, the FQHE is modified and new incompressible 
ground states should be conjectured in graphene |13H17l |. 
Moreover, the linear energy dispersion also modifies the 
inter-electron interactions, which implies a specific de- 
pendence of t he g round state energy and energy gap on 
LL index [13l - [l5l ]. The graphene- like integer quantum 



Hall effect has been observed in strained bulk HgTe [18 1 
and the LL spectrum has also been measured in three- 
dimensional topological insulator (TI) material Bi2Se3 
by using scanning tunneling microscopy [ij., _2(| . In ad- 
dition, features in the Hall resistance at fractional filling 
factors have been speculated to be related to the FQHE 
of TIs dH-il]. However, the strength of the FQHE will 
be different from that in graphene because the LLs of the 
two surfaces of the TI thin film can mix with each other 

Furthermore, previous studies suggest that when the 
spin freedom and the parallel magnetic field are taken 
into account, the spin- reversed quasiparticle (quasihole) 
excitations could be observed in certain FQHE gaps of 
GaAs-based quantum wells 0, __H • The quasiparticle- 
quasihole energy gap increases with increasing the tilt 
angle of magnetic field, and the electron-hole symme- 
try are broken [26|, H3] due to the subband-LL coupling 
in FQHE. The contrasting behavior of the higher filling 
{v = 5/2 and v = 7/3) FQHE states and the relevance 
of a skyrmion spin texture at v = 5/2 associated with 
small Zeeman energy in wide GaAs/AlGaAs quantum 
wells under a tilted magnetic field have been studied in 
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FIG. 1: (Color online) (a) The fermion Landau levels on the 
topological insulator surface under an external magnetic field. 
The corresponding experimental set sketch is shown in (b). 



recent experiments 28, 29]. Surprisingly, with increasing 
the tilt angle the FQHE states with even-denominator 
filling factors may transform to the compressible Fermi 
liquid state from the incompressible state. 

Unambiguous characterizations of the FQHE of Dirac 
fcrmions on TI surface (or graphene) in the presence of 
a tilted magnetic field, however, is still missing in litera- 
ture. Owing to the unique spin chirality of Dirac fermions 
induced by intrinsic strong spin-orbit coupling in TI ma- 
terials, the effect of tilted magnetic field on the FQHE on 
TI surface should remarkably differ from that in conven- 
tional 2DEG. Therefore, because of its importance both 
from basic point of interest and to the analysis of uncon- 
ventional properties of Tl-based FQHE, in the present 
paper we address the stability of the FQHE on TI un- 
der a strong tilted magnetic field by presenting an at- 
tempt at the theoretical evaluation of the effective pseu- 
dopotentials of the electron-electron interactions and the 
ground (excitation) state nature at 1/3-FQHE at lowest 
LLs problems. We will show that by increasing the in- 
plane component of the tilted magnetic field, the FQHE 
state at n=0 LL becomes more stable, while the situation 
for the n—±l LLs just have an opposite trend, becom- 
ing weaker. Besides, we find that the excitation gaps of 
the v = 1/3 FQHE states increase as the tilt angle is 
increased. 
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The single particle low-energy effective Hamiltonian of 
TI surface in the presence of a tilted magnetic field, B = 
B± + B\\ = B(xsin9 cos (j> + y sin 9 sin (f>+z cos 9), where 9 
is the tilt angle and tp is the azimuthal angle with respect 
to two-dimensional (2D) plane, is expressed as 



Ho = vfo ■ IT + g/J-B& ■ B, 



(1) 



where v F — 3 x 10 5 m/s is the Fermi velocity and 
IT = p + eA/c is the 2D canonical momentum. We 
choose the symmetric gauge for the vector potential of 
vertical magnetic field A± = B±(~y/2,x/2,0), and the 
Landau gauge for the vector potential of in-plane mag- 
netic potential Au = (zB y , — zB Xl 0). Because the mo- 
tions of electrons are confined in z — plane, the elec- 
tron canonical momentum is only relevant to Aj_. After 
introducing the ladder operators a 1 = ^ (Hi — 2Zc? R ) , 



x/2 V2Z 



2W K ) with k 



iy and the magnetic 



length I = J -4p- , we can rewrite the Hamiltonian (JTJ) as 



gH B B±_ 



-gH B B± 



(2) 

The isotropic property of electrons in 2D plane requires 
that the physical results should be independent of the az- 
imuth angle <j>. One can see in the following text that the 
Haldane's pseudopotentials rightly satisfy this symmetry 
condition. By employing the perturbation method and 
only keeping the first-order approximation of the wave 
function, the eigenstates of Hamiltonian ^ may be writ- 
ten as 
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(3) 



where 4> nm is the eigenstate of the 2D Hamiltonian with 
non-relativistic quadratic dispersion relation in nth LL 



with angular momentum m. Here, 



0, 

— isgn(n) cos $ n 
^2(l-sgn(n) sintf^) 

i, 



n=0 
, n^O 



n=0 



v /(l-sgn(n) sinz?„)/2, n^O 



(4) 



gn B B± 



where $„^o =tan 



/2\n\hv F l- 1 

efficients are expressed as AL 



A 2 , 
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The other co- 
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— e 2 _ & 2_ c 2(|„| + 1 ) , 
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with 6 = g^sB±, d = 

g^BB n e~ 1 ^ , c = \/2hv F l~^ '■ The corresponding eigen- 
values are given by Eq = —gfJ-BB± = —b, and 



e n ^o =sgn(n)y {g^ B B^) +\^/2\n\hv F l- 1 \ 2 . In the 
limit of .B||=0, the Hamiltonian (2) can be analytically 
solved (33-112 ■ 

In order to investigate the properties of TI surface 
FQHE, we first consider Coulomb interaction V(r) = 
e 2 /er between two electrons in nth LL with relative an- 
gular momentum m. The Haldane's pseudopotential is 
given by 



V 



(n.m) _ 1 \ ^ 



2ne A 



2^ eq 
q 



[T n {q)] 2 L m (q 2 l 2 )e-^-, (5) 



where L m (x) are the Lagueree polynomials. The form 
factor F n (q) in Eq. (O is given by 

MQ) = {KJe-^KJ + iKnle^Kmh (6) 



where n=r — R is the cyclotron variable with guiding- 
center R. The explicit expression of J^niq) is derived to 
be written as 



MQ) = KIM 2 + \Al\ 2 ) L|„| (| S | 2 ) + \B 2 \ 2 L lnl+1 (\s\ 2 ) 
+ {\a n \ 2 + \Blf) L |nR (| S | 2 ) + \A 2 n \ 2 L H _ 2 (\s\ 2 ) 

+a* J 4„X|„|_ 1) | n |_ 2 (s) +a„A n *X| n |_ 2 ,|n|-i(s) + A n A n * X\ n \_ 2t \ n \(s) + A]? A 2 n X\ n \^ n \_ 2 {s) 

+^n S n^M,|n| + l( s ) + Pn B n X \n\+l,\n\ ( s ) + B n B n X \n\-l,\n\+l{ s ) + B ^ B n X \n\ + \,\n\-\{ s ) 

+ «A x n + p n B x *) X lnl _ Un[ (s) + (a n A]: + p* n Bl) X HM _ x {s)] e^, (7) 
where X n , n ,( S )=X*, = (-«)»'- E^ n,n#) |^§g p| with s = [fa + 



A typical way to predict the stability of the FQHE is to consider only m=l and m=3 pseudopotentials, since 



3 




0.0 0.3 0.6 0.9 1.2 



FIG. 2: (Color online) The ratio of the first and third rel- 
ative angular momentum pseudopotentials for the Coulomb 
interaction in the n=0 LL (a), n=l LL (b), and n=— 1 LL 
(c) as a function of B„/B_i_. The black sold, red dashed, and 
blue dotted lines correspond to 7 = gfj,BB±/(HvFl~ 1 )=Q.l, 
0.2, 0.3, respectively. 



may be observed when the ratio between m=l and m—3 
pseudopotentials is larger than 1.3—1.5, because the com- 
posite fermion wavefunction is accurate when the inter- 
action is short range, and remains a good approximation 

when Kff'^/Kff'^ is large enough [7|. For the purpose 
of studying the stability of the FQHE states under a 
tilted magnetic field, we also employ this crucial crite- 
rion and choose 1.5 as a critical value. In other words, 
when V^S' 1 ^ /V}£' 3 ' is smaller than 1.5, we conclude the 
FQHE state can not be observed. 

For simplicity, let us first consider the stability of 
the FQHE state at n = LL, whose form factors 

is Ml)= [Lo (M 2 ) (N 2 ) e"^ . Sub- 

stituting this .Fo(g) into Eq. ([3]), one easily obtain the 
Haldane's pseudopotential for the 0-th LL, 
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FIG. 3: (Color online) Exact energy per electron versus the 
angular momentum L for N = 7 electrons at v — 1/3 FQHE 
states for (a) n — 0, (b) n = 1 and (c) n = — 1 LLs. We 
choose the parameter gfj,BB±/(hvpl~ 1 )=0.2. The dots and 
stars correspond to Bn=0 and B„=B± respectively. 
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falls off quickly as m increases. The FQHE state 
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One can find that, differing from the case under a fully 
perpendicular magnetic field, the form factor in a tilted 
magnetic filed case is dependent on the in-plane compo- 
nent of the external magnetic field. The numerical result 
of V^g /V^g' 3 ^ as a function of in- plane component of the 
external magnetic field B u is shown in Fig. 2(a), where 
different curves correspond to different given perpendicu- 
lar components of the external magnetic field B±. From 
Fig. 2(a) one can see that V^jj'^/V^g always increases 
with increasing B„. Based on this fact and on that the 
minimum value of V}g^/ Vgg is ~1.6 at B„=0 (larger 
than the critical value 1.5), we can immediately conclude 
that the FQHE state at n — LL is more stable by in- 
creasing the in-plane component of the tilted magnetic 
field. 

Now we turn out to study the cases for the n=±l LLs, 
whose pseudopotentials can be written as 



J 



V. 



(n— ±1 ,ra) 



err 
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1 - 



£ 2 (M 2 ) 



(9) 



where |< ow | 2 = \a n \ 2 (l + ||| 2 ), and \(3»™\ 2 = 

|/3„| 2 (l + |^| 2 |f | 2 )- Figures 2(b) and 2(c) respec- 
tively plot V^ -±1 ' 1 ''/KfT -±1 ' 3 ' > versus B„, from which 



one can find that V^g^ 1 ' 1 ^ /V^2 _±1 ' 3 - ) always decrease 
with increasing £?,,. These results imply that the Hal- 
dane's pseudopotentials for the n=±l LLs the FQHE 
states turn to become instable by increasing the in-plane 
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FIG. 4: (Color online) The gap width between the ground 
state and the first excited state at v — 1/3 FQHE states 
for LLs n = and ±1 as a function of B///B±. The other 
parameters are N = 7 and gfiBB±/(hvFl~ 1 )=0-2. 

component of the tilted magnetic field, which is quite 
different from the case for n=0 LL. 

We also investigate the energy spectra of the many- 
body states at fractional filling v = 1/3 of the LLs n—Q 
and ±1 under a tilted magnetic field by numerically di- 
agonalizing the many-body Hamiltonian in the spherical 
geometry. Figure 3 shows the energy spectra for N = 7 



electrons at v = 1/3 FQHE states for n=0 and 1 LLs with 
different in-plane components of the external magnetic 
field, -B||=0 (black dots) and Bn=B± (red stars). One 
can clearly see that the excitation gap width at B\ \ =B± 
is larger than that at -B||=0. To more clearly see how the 
gap width changes with the tilt angle, in Fig. 4 we exhibit 
the gap width between the ground state and the lowest 
excited state as a function of the in-plane component of 
the magnetic field. It is obvious that by increasing the 
in-plane component of the magnetic field, the gap widths 
at v = 1/3 FQHE states for LLs n~0 and ±1 become 
larger and larger, which is similar to the conventional 
2DEG cases P,^- 

In summary, we theoretically investigated the FQHE 
in TIs under a tilted strong magnetic field. The sin- 
gle particle wave function was obtained by using a sim- 
ple perturbation method. The effective pseudopoten- 
tials of the electron-electron interactions and the ground 
(excited) state energy spectra for 1/3-FQHE at lowest 
LLs were calculated within the exact diagonalization ap- 
proach. We have shown that in the presence of a in-plane 
component of the tilted magnetic field, the FQHE state 
at n=0 LL becomes more stable, while the stabilities of 
n=±l LLs become weaker. Moreover, we have also found 
that the excitation gaps of the v = 1/3 FQHE states in- 
crease as the tilt angle is increased. 
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